Visualization of Repetitive Patterns in Event Traces

Performance Tracing has always been challenged by large amounts of trace data. Software tools for trace analysis and visualization successfully cope with ever growing trace sizes. Still, human perception is unable to “scale up” with the amounts of data. With a new model of trace visualization, we try to provide less data but additional information or rather more convenient information to human users. By marking regular repetition patterns and hiding the inner details, less complex visualization can offer better insight. At the same time potentially interesting irregular sections are revealed. The paper introduces the origin of repetition patterns and outlines the detection algorithm used. It demonstrates the new visualization scheme which has also been incorporated into Vampir NG as a prototype. Finally, it gives an outlook on further development and possible extensions. 1 Performance Tracing Event tracing is a well-established method for performance analysis of computer programs, especially in parallel and High Performance Computing (HPC) [5,9]. It has a reputation for producing large amounts of trace data where “large” has always been defined by the time’s standards. Many advancements in HPC contributed to that. This includes faster processors and growing parallelism. Also, more detailed instrumentation and additional data sources increase trace data volume. Last but not least availability of larger memory and storage capacities allowed traces to grow. Therefore, trace analysis and visualization has always been a challenging task on contemporary computers and always will be. The more so as one cannot require an actual super-computer for analyzing super-computer traces. But what is the effect from the human users’ perspective? Could screen resolution grow with an appropriate rate? Can human perception scale with the growing amounts of data as well? And is there really more information when there is simply more data1? 1 Let “information” be what contributes to the users insight while “data” is just the byte sequence transporting information. B. K̊agström et al. (Eds.): PARA 2006, LNCS 4699, pp. 430–439, 2007. c © Springer-Verlag Berlin Heidelberg 2007 Visualization of Repetitive Patterns in Event Traces 431 The next chapter tries to address those questions. Then, Section 3 will propose a new visualization methodology to provide more convenient information or to the user with less data. The following Section 4 will outline the algorithms for pattern detection. In the final Section 5 there is a conclusion as well as an outlook on improved analysis methods based on patterns. 2 Data vs. Information of Traces There are several reasons for growing trace sizes. First, there may be larger software projects with bigger source codes. Also, it might grow larger due to optimization and specialization of code. Second, instrumentation and measurement evolve, providing additional data, for example performance counters in processors, in communication sub-systems and in I/O backends. Third, longer or faster running programs with more and more parallelism will increase the number of repeated executions of certain program parts. The former two reasons have only secondary effect on growing trace data sizes. They will hardly increase it by orders of magnitude. Only the latter is responsible for traces of tens of gigabytes, today. On average the trace size will double when iteration count or run-time is doubled or when there are twice as many parallel processes. (This involves a certain redundancy which is exploited by approaches like introduced in Section 4.1.) In terms of information to the user the same situation looks different. Assuming an interesting situation can be described as “sequence A = (a1, a2, . . . ) is repeated n times”. That might have been inferred from a Vampir display showing distinguishable iterations like in Figure 2. How would the statement “sequence A is repeated 4n times” relate to it? (Compare Figure 1) Would this fact be four times as useful to the human user? Or would it carry an “equal amount” of information? Or even less information because it might not be distinguishable anymore with limited screen resolution? Thus, information is difficult to quantify, although data size is evidently fourfold. Fig. 1. Vampir screen-shot of 32 identical iterations hardly distinguishable with current horizontal resolution. Note that iterations are of strictly regular structure unlike runtime which shows some notable delays. 432 A. Knüpfer et al. Fig. 2. Vampir screen-shot showing a subset of 8 iterations from above (Figure 1). Only with this zoomed version iterations’ structure becomes visible. However, hidden in the data there are some additional information of interest. First, whether there is a (strict) regular standard behavior among iterations. Most likely this is independent of iteration count. Second, if there are outliers and abnormal cases differing from regular behavior. This additional information is not accessible to the human user by pristine visualization. Rather sooner than later human perception will be overcharged by too much data. Could one tell for the iterations shown in Figures 1 and 2? Therefore, we want to propose a new scheme of visualization that reduces the amount of data in favor of information perceivable by human users. Nevertheless, all familiar information and data will still be available on demand. 3 Visualization of Repetition Patterns The new visualization approach focuses on Process Timeline Diagrams like found in the Vampir NG [1] tool, for example. For each and every function call a rectangle represents a state of execution, compare Figure 3a. Now, in a simplified Process Timeline Diagram all regular repetition patterns of arbitrary size and nesting depth are replaced with a single highlighted rectangle. See Figure 3b for an example. Those rectangles indicate a region with regular behavior, inner details are not shown.